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We suggest to simulate evolution of complex organisms constrained by the sole requirement of 
robustness in their expression patterns. This scenario is illustrated by evolving discrete logical 
networks with epigenetic properties. Evidence for dynamical features in the evolved networks is 
found that can be related to biological observables. 
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I. INTRODUCTION 

A common concept in evolution is fitness and fitness 
landscapes |IJ , and often evolution is viewed as hill climb- 
ing, possibly with jumps between fitness maxima 
However, fitness landscapes implicitly assume that fitness 
is varying over a well-defined metric in genomic space. 
This would be the case if single point mutations were 
a driving force. However, significant genome rearrange- 
ments are observed already in the rather brief real-time 
evolution experiments of Escherichia coli cultures of Pa- 
padopoulos et al. 0. Genomic rearrangements short- 
circuit the simple metric generated by one point muta- 
tions, usually underlying the intuition of evolution on 
landscapes. As a consequence the combinatorial distance 
for moving from a genome A to a genome B may easily be 
different from the distance of the opposite move, simplest 
exemplified by deletions and insertions. Thus, although 
fitness landscapes have a meaning for the small scale ad- 
justments associated to fine-tuning of binding constants, 
it is an unjustified concept for evolutionary changes on 
the scale of speciation events. 

Abandoning fitness landscapes we here instead discuss 
the possibility that evolution progresses through a pro- 
cess where genotypes and phenotypes subsequently set 
the frame at which the other may change. Of particular 
relevance for this view of evolution is the fact that one of- 
ten observes different phenotypes for the same genotype. 
This viewpoint is in part supported by cell differentia- 
tions within one organism, in part supported with epi- 
genetics and the large class of organisms which undergo 
metamorphosis and thus exist in several phenotypes for 
the same genotype. Recently, it has also been proposed 
that genotype-phenotype ambiguity is governing spe- 
ciation events. 

A class of systems that exhibits epigenetics is repre- 
sented by the logical networks, where nodes in the net- 
work take values on or off, as function of the output of 
specified other nodes. This has been suggested to model 
the regulatory gene circuits |^-^| where specific genes 
may or may not be expressed as function of other genes. 
In terms of these models it is natural to define genotypes 
in form of the topology and rules of the nodes in the 
network. The phenotypes are similarly associated to the 
dynamical expression patterns of the network. 

To define the rules under which phenotypes and geno- 



types set the frame for each other's development, a model 
for evolution should fulfill the requirement of robustness. 
Robustness is defined as the ability to function in face 
of substantial change in components |j-12|. Robustness 
is an important ingredient in simple molecular networks 
and probably also an important feature of gene regula- 
tion on both, small and large scale. In terms of logi- 
cal networks, robustness is implemented by constraining 
subsequent networks to have similar expression patterns. 

This article is organized as follows: First we discuss dy- 
namics on logical networks and numerically review the 
basic properties of attractors of random threshold net- 
works and Boolean networks. Then we propose a mini- 
mal evolution model and investigate its statistical and 
structural implications for the evolved networks. Fi- 
nally biological implications, and possible experimental 
approaches to the dynamics of real genetic networks are 
discussed. 



II. DYNAMICS ON LOGICAL NETWORKS 

Let us first discuss two prototype networks that ex- 
hibit epigenetics, Boolean networks and threshold 
networks jl3| ]. These are both networks of logical func- 
tions and share similar dynamical properties. We here 
briefly describe their definition and dynamical features. 
In both networks each node is taking one of two discrete 
values, ±1, that at each timestep is a discrete function 
of the value of some fixed set of other nodes specified by 
a wiring diagram. If we denote the links that provide 
input to node i by {iOy}, with Wy = ±1 also, then for 
the threshold network case the updating rule is additive 
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In the Boolean network case the updating is a general 
Boolean function of the input variable 



<7j = Bicij's which provide input to i). 



(3) 



Thus, the threshold networks form a hugely restricted set 
of the Boolean networks. Boolean networks include all 
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nonlinear combinations of input nodes, including func- 
tions as, for example, the "exclusive or" . 

The basic property of logical networks is a dynamics 
of the state vector { oi } characterized by transients that 
lead to subsequent attractors. The attractor length de- 
pends on the topology of the network. Below a crit- 
ical connectivity K c ~ 2 the network decouples 
into many disconnected regions, i.e., the corresponding 
genome expression would become modular, with essen- 
tially independent gene activity. Above K c any local 
damage will initiate an avalanche of activity that may 
propagate throughout most of the system. For any K 
above K c the attractor period diverges exponentially 
with respect to system size N and in some interval above 
K c the period length in fact also increases nearly expo- 
nentially with connectivity K ]T^ |. 



III. STRUCTURAL EVOLUTION OF 
NETWORKS 

Dynamics may occur on networks as defined by the rule 
above, but at least as important is the dynamics of net- 
work topology. In terms of network topology an evolution 
means a change in the wiring {Wij} — > {w^} that takes 
place on a much slower timescale than the {<Jj} updating. 
The evolution of such networks represents the extended 
degree of genetic network engineering that seems to be 
needed to account for the large differences in the struc- 
ture of species genomes [Q, given the slow and steady 
speed of single protein evolution jl7j . 

We have in an earlier publication proposed to evolve 
Boolean networks with the sole constraint of continuity 
in expression pattern [|TJ. Here we simplify this model 
by simple damage spreading testing: 

The model evolves a new single network from an 
old network by accepting rewiring mutations with 
a rate determined by expression overlap. 

This is a minimal constraint scenario with no outside 
fitness imposed. Further the model tends to select for 
networks which have high overlap with neighbor mutant 
networks, thus securing robustness. 

Now let us formulate an operational version of the evo- 
lution in terms of threshold networks as these have com- 
parable structural and statistical features to the Boolean 
ones [[[3). Consider a threshold network with N nodes. 
To each of these let us assign a logical variable cr, = — 1 or 
+1. The states {<7i} of the ./V nodes are simultaneously 
updated according to (1) where the links Wij are specified 
by a matrix. The entry value of the connectivity matrix 
Wij may take values —1 and +1 in case of a link between 
i and j, and the value if i is not connected to j. 

The system that is evolved is the set of couplings Wij 
in a single network. One evolutionary time step of the 
network is: 



1) Create a daughter network by a) adding, b) remov- 
ing, or c) adding and removing a weight in the coupling 
matrix Wij at random, each option occurring with prob- 
ability p = 1/3. This means turning a Wij — to a 
randomly chosen ±1 or vice versa. 

2) Select a random input state {cr.;}. Iterate simulta- 
neously both the mother and the daughter system from 
this state until they either have reached and completed 
the same attractor cycle, or until a time where {c^} dif- 
fers between the two networks. In case their dynamics 
is identical then replace the mother with the daughter 
network. In case their dynamics differs, keep the mother 
network. 

Thus, the dynamics looks for mutations which are phe- 
notypically silent, i.e., these are neutrally inherited under 
at least some external condition. Notice that adding a 
link involves selecting a new Wy, thus changing the rule 
on the same timescale as the network connectivity. Iter- 
ating these steps represents an evolution which proceeds 
by checking overlap in expression pattern between net- 
works. If there are many states that give the same 
expression of the two networks, then transitions between 
them are fast. On the other hand, if there are only very 
few states {<7i} which result in the same expression for 
the two networks, then the transition rate from one net- 
work to the other is small. If this is true for all its neigh- 
bors then the evolutionary process will be hugely slowed 
down. 

In Fig. 1 the connectivity change with time for a 
threshold network of size N = 32 is shown. 
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FIG. 1. Long time evolution for the connectivity of of a 
threshold network with N=32 nodes. Connectivities are con- 
strained to be below K — 8. One observes long periods of 
stasis interrupted by sudden changes, reminiscent of punctu- 
ated equilibrium. 

Time is counted as number of attempted mutations, 
and one observes that especially for high connectivity the 
system may stay long time at a particular network before 
an allowed mutation leads to punctuations of the stasis. 
The overall distribution of waiting times is ~ l/i 2±0 - 2 . 

One feature of the evolution is the structure of the 
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evolved networks, which can be quantified by the average 
length of attractors for the generated networks. This is 
shown in Fig. 2, where they are compared with attractor 
lengths for random networks at the same connectivity. 
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FIG. 2. Average length of periodic attractors for evolved 
and for random networks. Also the periods of the unsuccessful 
mutations in the presence of newly chosen random initial con- 
ditions are shown, demonstrating that selection of networks 
is indeed operating in structure space and the specific input 
configuration in the event of selection does not play a major 
role. 

One observes that the evolved networks have much 
shorter attractors than the random ones, thus our evolu- 
tion scenario favors simplicity of expression. 

To examine further the expression behavior of the net- 
works let us consider the size of frozen components as 
introduced by Kauffman for Boolean networks 0. A 
frozen component is the set of nodes connected to a given 
attractor that does not change at any time when you it- 
erate along the attractor, i.e., a frozen component rep- 
resents genes which are anesthesized under a given at- 
tractor/initial conditions. In Fig. 3 one sees that the 
frozen component for the evolved network typically in- 
volves half the system, and thus is much larger than the 
typical frozen component associated to attractors of ran- 
domly generated threshold networks. Also we test frozen 
components for random one mutant neighbors of the se- 
lected ones, and find that these networks also have huge 
frozen components. 

Let us finally look at the active part of the network and 
the complexity of its expression pattern. As a quite large 
fraction of the nodes may belong to the frozen component 
of the network, the remaining active part of the nodes 
may behave differently from the average dynamics of the 
whole network. One possible measure is the number of 
times, each non-frozen node switches its state during the 
dynamical attractor. 
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FIG. 3. Average size of frozen components as a function 
of connectivity for evolved and random networks. The frozen 
component is the set of all nodes that do not switch during 
the attractor. One observes that the robustness constraint in 
evolution favors a larger frozen component. 

In Fig. 4 this quantity is shown for random networks 
as well as evolved networks. One observes that the ac- 
tive part of the evolved networks exhibits a much sim- 
pler expression pattern than that of a random network 
of comparable connectivity. 
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FIG. 4. Average number of flips per node in the non-frozen 
part of the network, as a function of connectivity for evolved 
and for random networks. The evolved networks show a re- 
duced activity in the non-frozen nodes resulting in simple ex- 
pression patterns as compared to those of random networks 
of same connectivities. Notice that the number counts off-on 
and on-off transitions of the nodes as separate events. 

Overall, requiring robustness as an evolution criterion 
has observable consequences for both, the temporal evo- 
lution pattern, and for confining possible genetic network 
architectures to the ones with simple expression patterns. 
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IV. DISCUSSION 

Some quantitative testing of the minimal evolution sce- 
nario is possible on the macro-evolutionary scale. Here 
the intermittent evolution of the networks bears resem- 
blance to the punctuated equilibrium observed for species 
in the fossil record |^2f . Quantitatively the 1/f power 
spectra and 1/t 2 stability distribution for single networks, 
that one finds for this model as well as for the earlier 
version pjj , compares well with the similar scalings ob- 
served for the statistics of birth and death of individual 
species in the evolutionary record |l8|,[l9| . Obviously the 
here ignored features related to co-evolution prevent us 
from discussing co-extinctions |lq |. In fact the analogy 
can even be fine-grained into a sum of characteristic life- 
times, each associated to a given structural feature of the 
networks plf . A similar decomposition is known from 
the fossil record |2(| , where groups of related species dis- 
play Poisson distributed lifetimes and therefore similar 
evolutionary stability. 

A validation on the microlevel based on statistical 
properties of genetic regulatory circuits has to be based 
either on properties of genetic networks or on evolu- 
tion and mutation experiments of fast lived organisms as 
E. coli Q . A key number is the estimated average connec- 
tivity K of 2 3 in the E. coli genome [^3| . Information 
on the overall organization of these genetic networks is 
obtained from gene knock out experiments. 

A quantitative support for a connected genome can 
be deduced from Elena and Lenski's H| experiments on 
double mutants, which demonstrated that about 30-60% 
of these (dependent on interpretation) change their fit- 
ness in a cooperative manner. In terms of our networks, 
we accordingly should expect a coupled genetic expres- 
sion for about half of the of pairs of genes. Although our 
evolved networks can give such correlations for the con- 
nectivity estimate of 2-3 given by |23|, the uncertainty 
is still so large that random networks also are in accor- 
dance with data. Further one should keep in mind that 
the E. coli genome is large and not well represented by 
threshold dynamics of all nodes, and also that only be- 
tween 45 and 178 of the E. ca/i's 4290 genes are likely 
to mediate regulatory functions p5|| . Thus, most of the 
detected gene-gene correlations presumably involve genes 
which are not even regulatory, but instead metabolic and 
their effect on each other more indirect than in the case 
of the regulatory ones. Presumably one would obtain 
stronger elements of both coupling and correlation if one 
specialized on regulatory genes. Thus one may wish for 
experiments where one and two point mutations are per- 
formed in regulatory genes only. A more direct test of 
our hypothesis of damage control as a selection criterion 
may be obtained from careful analysis of the evolution of 
gene regulation in evolving E. coli cultures. 

Another interesting observation is the simplicity of bi- 
ological expression patterns. For example as observed in 
yeast many genes are only active one or two times dur- 



ing the expression cycle pq] , thus switching from off to 
on or on to off occurs for each gene in this system only 
a few times during expression. For random dynamical 
networks of comparable size one would expect a much 
higher activity. Thus surprisingly simple expression pat- 
terns are observed in biological gene regulatory circuits. 
This compares well with our model observation where 
simplicity of expression patterns emerges as a result of 
the evolutionary constraint. 



V. SUMMARY 

In this article we have proposed a computer simula- 
tion of evolution operating on logical networks. The 
scenario mimics an evolution of gene regulatory circuits 
that is governed by the requirement of robustness only. 
The resulting dynamics evolves networks which have very 
large frozen components and short attractors. Thus they 
evolve to an ordered structure that counteracts the in- 
creasing chaos when networks become densely connected. 
The evolved architecture is characterized by simplicity of 
expression pattern and increased robustness to perma- 
nent mutational fluctuations in the network architecture 
- features that are also seen in real molecular networks. 
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